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I.  Introduction 


'Hiis  task  coiUiiuics  work  pciioriuccl  iiiulcr  a  previous  task  (I)AAI  10 ! -9  I -13-K()().S  DO.  75)  entitled 
“  rrainer  Methodology  Study’’  and  documented  in  a  report  ot  the  same  name.  I  hat  report  documented  two 
software  programs  called  PathEdit  and  Depth  Finder  which  are  used  in  creating  scenarios  for  a  training  device 
called  the  Basic  Skills  Trainer,  or  13ST,  which  displays  moving  targets  overlaid  on  bitmapped  terrain  images. 
The  terrain  images  for  the  existing  BST  being  used  in  the  field  at  this  time  are  artificially  generated  and  not 
particularly  realistic.  One  aspect  of  the  ongoing  BST  work  is  to  develop  methods  ol  using  pliotographs  of 
lei  ram  as  the  backdrop  to  provide  better  realism.  To  create  BST  scenarios  using  photographs  requires  a  means 
1)1' defining  range  and  relative  depth  information  for  objects  in  the  scene,  1  he  range  inlornuuion  is  needed  for 
scalinu  targets;  the  relative  depth  information  for  occluding  targets.  1  he  Depth  Finder  program  is  intended  as 
a  tool  for  generating  that  range  and  depth  inlbrmalion.  As  discussed  in  the  previous  report  it  includes  a  camera 
calibration  procedure  for  determining  values  of  camera  parameters  and  a  procedure  lor  extracting  true  3D 
coordinates  of  objects  in  the  scene  relative  to  the  camera  Irom  a  stereo  pair  ol  photographs.  I  lie  latter 
procedure  depends  on  good  parameter  values  from  the  lormcr  procedure. 

[>art  of  the  current  task  has  been  an  attempt  to  simplify  and  standardize  the  calibration  procedure. 
However  a  difficulty  has  emerged  in  obtaining  camera  parameters  from  close  range  photos  taken  for  calibration 
purposes  and  applying  those  parameters  to  long-range  terrain  photos.  I  his  is  believed  to  be  the  result  ol  the 
change  of  focus;  so  far  the  work  has  been  performed  using  an  auto  locus  digital  camera  (kodak  DC5())  which 
has  no  means  of  maintaining  a  fixed  focus  setting  Irom  one  pliotogi’aph  to  another.  Section  2  ol  this  repot  t 
documents  the  calibration  work  for  this  task. 

In  order  to  pi  ovitie  a  means  of  generating  the  relative  depth  data  needed  by  the  existing  BS  I  witlunil 
stereo  photographs  and  accurate  camera  calibration  parameters,  a  second  program  called  Depth  Editor  has 
been  developed  as  part  of  this  task.  This  program  allows  manual  editing  ol  the  8-bil  depth  image  used  in  the 
existing  LiST  system.  The  Depth  Editor  program  is  documented  in  section  3. 


2.  Camera  Calibration 


rhe  material  in  the  previous  report  entitled  '  i  rainer  Methodology  Study"  is  a  prerequisite  lor 
understanding  this  section,  particularly  section  3.3  of  that  report  on  camera  geometry  and  c<ilihralion.  'fhe 
parameters  of  interest  are  scale  parameters  c,  and  r:  ,  translation  parameters  Xq"  and  yo'\  and  lens  distortion 
parameters  k„  and  k,.  These  seven  parameters  are  together  called  interior  orientation  parameters.  They 
relate  pixel  coordinates  in  the  terrain  image  to  virtual  photo  coordinates  (see  previous  report)  On  page  19  of 
the  previous  report  it  was  stated  that: 

“Currently  it  appears  that  these  interior  orientation  paiameters  are  consistent  enough  across  vaiious  photos 
taken  with  the  same  camera,  that  wc  can  treat  these  parameters  as  being  camera  constants. 

'fhis  statement  may  in  fact  iiold  true  for  a  digital  camera  with  a  manual  focus  setting  left  in  a  lixcd  position 
between  photos.  However,  at  present  all  work  has  been  perlormed  using  a  digital  camera  (Kodak  DC50)  which 
docs  not  allow  the  auto  locus  system  to  be  overridden.  Using  this  camera  it  has  become  apparent  that 
acceptable  results  cannot  be  obtained  by  calibrating  the  camera  at  close  range  and  tlien  applying  the  results 4o 
long  range  photos. 

Although  the  calibration  results  did  not  turn  out  to  be  acceptable  in  the  sense  just  discussed, 
improvements  in  the  calibration  procedure  developed  during  this  task  hopefully  will  be  applicable  to  the  fixed 
focus  case.  Hiis  will  be  te'sted  in  the  near  future.  In  the  meantime,  since  the  investigation  is  continuing,  the 
documentation  ol'the  improvements  to  the  procedure  given  here  will  be  brief. 

'Uic  calibration  procedure  depends  on  the  use  of  control  point  coordinates.  C:onlro!  points  are  points 
with  known  3d  positional  coordinates  in  some  coordinate  s)'stem  external  to  the  camera.  In  our  case,  the  points 
are  the  intersections  ol'  lines  drawn  on  poster  board  and  lorming  a  regular  grid  (see  figure  I ).  As  described  in 
the  previous  report,  the  least  squares  ad  justment  which  computes  values  of  the  interior  orientation  parameters 
simulaneously  adjusts  the  values  of  six  exterior  orientation  parameters.  These  exterior  orientation  parameters 
consist  of  three  translation  values  and  three  rotation  values,  which  together  describe  the  relationship  ol  the 
camera  coordinate  system  to  the  external  coordinate  system  in  which  the  control  points  have  known 
coordinates.'  In  our  case  the  external  coordinate  system  has  two  lines  of  the  grid  as  its  x-  and  y-axes  and  the 
line  perpendicular  to  the  poster  lioard  at  the  intersection  ol  these  two  lines  as  its  z-axis. 

It  turns  out  (hat  the  least  squares  adjustment  does  not  converge  correctly  if  all  the  control  points  lie  in  a 
single  plane,  f  or  this  reason  two  poster  boards  with  grid  lines  placed  at  an  angle  to  one  another  are  used.  One 
uf  the  poster  boards  forms  the  basis  for  the  external  coordinate  system.  W)  follow  through  the  calibration 
procedure  as  described  so  far,  the  coordinates  of  the  grid  points  on  the  second  board  need  to  be  determined 
relative  to  the  first  board.  b>'  manual  measurement  or  some  other  means,  lo  get  around  (his  task,  new 


Figure  1 :  Calibration  Grid  and  Adjiislmcnt  Dialog 

adjuslmcnt  equations  were  derived,  incorporating  as  unknowns  another  six  exterior  orientation  parameters  for 
the  second  poster  board.  This  allows  all  control  point  coordinates  to  be  given  in  their  natural  coordinate 
system;  i.c.  as  an  offset  from  a  grid  point  on  the  same  board  designated  as  the  origin  for  that  board.  Rough 
approximations  of  each  of  the  two  sets  of  exterior  orientation  parameters  arc  given  as  inputs.  Values  of  these 
jiarameters  appear  in  the  two  sets  of  six  readouts  toward  the  bottom  of  the  dialog  box  as  shown  in  [-igure  1. 
fhe  buttons  .labeled  ‘'Adjust"  allow  each  of  these  exterior  orientations  to  be  adjusted  independently,  with  the 
interior  orientation  parameters  fixed.  Another  button,  labeled  “Adjust  10/120  performs  the  adjustment 
described  in  the  previous  report  in  which  both  interior  and  exterior  orientation  parameters  change  to  most 
closely  fit  the  measurements  to  the  input  control  point  data. 


3.  The  Depth  Editor  Program 


The  Depth  Editor  progiani  is  used  lo  create  a  depth  image  corresponding  to  a  given  terrain  bitmap 
image.  The  two  images  are  the  same  size  and  resolution,  so  that  there  is  a  one-to-one  correspondence  between 
terrain  pixels  and  depth  pixels.  A  pixel  in  the  terrain  image  represents  the  visible  color  displayed  at  that  point, 
while  a  pixel  in  the  depth  image  represents  the  relative  depth  at  that  location.  The  depth  image  is  8  bits  deep, 
each  pixel  having  a  value  from  0  (closest  to  viewer)  to  255  (furthest  from  viewer).  Target  paths  specify  a  depth 
at  each  location  along  the  path,  and  the  BST  makes  pixcl-by-pixcl  depth  comparisons  to  determine  occlusion  of 
the  target  by  terrain  or  other  targets. 

The  first  step  in  using  Depth  Editor  is  to  o|ren  a  terrain  image  using  the  file  menu.  I'he  terrain  image 
must  be  a  standard  bitmap  file.  I'hc  corresponding  depth  image  to  be  created  or  edited  resides  in  the  same 
directory  as  the  terrain  image  aiui  has  the  same  name  except  for  the  extension,  which  is  .BMP  lor  the  terrain 
image  and  .DEP  for  the  depth  image.  If  no  corresponding  depth  image  exists  at  the  time  the  terrain  image  is 
opened,  the  program  presents  a  dialog  box  allowing  the  creation  of  a  baseline  depth  image  from  a  depth  table 
created  by  specifying  a  horizon  line.  A  depth  table  is  a  table  specifying  a  single  depth  value  for  each  horizontal 
line.  Thus  if  the  terrain  image  is  4096  pixels  wide  by  5 12  pixels  high,  the  depth  table  would  specify  a  depth 
value  for  line  0,  a  depth  value  for  line  1 ,  and  so  on  up  to  line  511. 


Figure  2:  Depth  Editor  Main  Window  and  Dialog 


Once  the  baseline  depth  iniape  based  on  a  depth  table  has  been  defined,  individual  regions  ol  a  fixed 
deplli  are  “painted  '  in  using  (Irawini:  tools  like  those  found  in  a  paint  program.  I  he  tools  consist  of  a  lieeliand 
"peiicir'  and  “eraser/'  a  line  tool,  rectangle  tool,  region  fill  tool,  and  edge  findei.  baich  of  these  tools  has  an 
icon  in  the  toolbar  at  the  top  and  a  corresponding  menu  item.  I  lie  toolbar  also  has  3  lines  ol  vaiying  width 
indicating  the  currently  selected  brush  size.  Vhc  tools  are  used  to  designate  a  .set  ol  pixels  as  being  selected, 
rhese  are  the  pixels  to  which  a  depth  will  be  applied  when  tlic.“ Apply  button  on  the  dialog  box  is  hit.  The 
depth  to  be  assigned  is  the  shown  in  a  field  on  the  dialog  box;  this  value  may  be  typed  in  by  the  user  or  taken 
from  the  depth  table  using  the  button  provided  for  that  purpose.  Hitting  that  button  causes  the  depth  value  from 
the  table  at  the  vertically  lowest  pixel  currently  selected  to  be  filled  in  to  depth  field,  where  it  can  then  be 
assigned  to  all  of  the  selected  pixels  by  hitting  (he  “Apply'’  button  as  indicated  above. 

Results  may  be  viewed  al  any  point  along  the  way  by  using  options  on  the  View  menu  to  display  the 
depth  data  in  place  of  or  overlaid  on  lop  oftlie  image  data.  The  depth  data  is  displayed  using  a  256-color 
palette  which  may  be  viewed  or  altered  using  the  “Depth  Palette”  dialog  box.  When  the  depth  image  is 
satisfactory  it  is  saved  using  the  “Save”  item  on  the  File  menu. 


